The present invention relates to communications between devices over a network. More particularly, the present invention relates to communications over a low-speed or wireless communication link between two computers using a message queue.
Traditional mainframe computer configurations provided for user interface to the computer through computer terminals which were directly connected by wires to ports of controllers connected by channels to the mainframe computer. As computing technology has evolved, processing power has typically evolved from a central processing center with a number of relatively low-processing power terminals to a distributed environment of networked processors. Examples of this shift in processing include local or wide area networks which interconnect individual work stations where each workstation has substantial independent processing capabilities. This shift may be further seen in the popularity of the Internet which interconnects many processors and networks of processors through devices such as, for example, routers.
At the same time that processing power was becoming more distributed there was also an increase in the popularity of mobile computing. The use of laptops, notebooks, Personal Digital/Communication Assistants (PDAs/PCAs) and other portable devices has led to an increase in demands for wireless communications. Wireless communication allows a user freedom to move within the wireless environment while remaining xe2x80x9cconnectedxe2x80x9d to a network. Furthermore, a wireless connection to a network allows a portable processor user the convenience of connecting to a network without having to plug into a docking station or use some other method of xe2x80x9chardwiringxe2x80x9d to a network. However, wireless wide area networks, cellular communications and packet radio, suffer from common limitations such as the high cost per byte of communications, slow response time, low bandwidth and unreliability which all hamper use of wireless technology.
Even outside of the portable processing arena wireless communications have seen an increase in popularity. Thus, as a result of infrastructure limitations, cost or convenience, it is becoming more frequent that a network linking two devices wishing to communicate may include a low through-put component such as a wireless network link.
Communications between devices such as computers are typically disadvantaged particularly on lower through-put network legs, such as wireless legs (or highly congested legs which are effectively bandwidth limited), where bandwidth limitations result in slower response time for communications between the computers. In the extreme, protocol timeouts may even cause transmission errors and resulting retransmissions or even inability of the communication system to operate. Thus, utilizing wireless technology, or any low-speed communication technology, for data communication between applications executing on computers exacerbates the weaknesses of the wireless technology.
Communications are further complicated where a plurality of applications executing on a computer perform operations requiring transmission of data streams over a shared port to an external communication link, on occasion with interruptions in session connections. It is known to provide for this shared access using asynchronous message queuing systems such as International Business Machine Corporation""s Message Queuing Series (xe2x80x9cMQSeriesxe2x80x9d), Telecommunications Access Method (xe2x80x9cTCAMxe2x80x9d), or Information Management Systems (xe2x80x9cIMSxe2x80x9d), which enable applications to queue data for transport (transmission) to a partner destination computer device on an external communication link such as a network. Using asynchronous message queuing, the external connection between the source computer and the destination computer typically does not have to exist at the time of submission of a data stream message for transmission and the source or sending device is not required to synchronously wait for a response from the destination device. In other words, the transmitting application hands over responsibility for the message to the transport queuing application which takes on responsibility for eventually delivering the queued message. In general, any data object, including messages, files, images, containers, etc., can be transported using a message queuing system.
The message queue transport application reads messages from the queue and sends them to the destination devices over the network. The communication protocol between transmitting and receiving message queues provides for assumption of responsibility over a transmitted message by the receiving device. The source message queue transport application typically then frees the queue space occupied by a message once confirmation is received that the receiving device has assumed responsibility for the message.
In view of the above limitations, it is one object of the present invention to provide for improved performance for data communications in a low-speed communication environment such as wireless communications.
It is a further object of the present invention to support such communications where the communicating applications may be only intermittently connected.
It is an additional object of the present invention to support such communications where the external communication link used for communication is shared by different applications.
In view of these and other objects, the present invention provides methods, systems and computer program products supporting differencing data communications using a message queue supporting asynchronous communications from a variety of applications executing on a source device over a shared external communication link to destination devices. At least one segment of a data stream between the application on the source device and an application on the destination device occurs over an external communication link. The present invention provides increased communications performance by combining data stream differencing with asychronous message transmission control using a message queue. The data stream is segmented based on the type of the data stream to provide a logical segmentation which increases the occurrence of repeated transmissions of a segment. Each segment is then placed in the message queue as a message for transport to a destination computer. Differencing is provided by replacing the segment with an associated identifier for segments which have previously been transported to provide a reduced volume of data for transmittal based on recognition and replacement of data segments which have previously been transmitted by the source device.
The intercept system on the receiving end at the destination computer receives the transmitted reduced segments as messages, expands the reduced segments and writes the expanded segments to a receive message queue. Segments are read from the receive message queue and combined to form the reconstructed data stream. The reconstructed data stream is then provided to the target application on the destination computer. By providing protocol conversion at both ends of the external communication link, the applications may continue to operate without any need to recognize the protocol differencing conversion provided by the present invention. Furthermore, synchronization is not required between the devices as the message queue provides asynchronous communications and, if a reduced segment is not recognized, retransmission of the complete segment using the segment identifier may be requested and the source device creates and queues the segment as a message.
In one embodiment of the present invention, a method is provided for differencing data communications using a message queue. A data stream having an associated type is intercepted (received) from a host application prior to transmission of the data stream through the message queue on an external communication link to a destination computer. The data stream is segmented based on the associated type to provide a plurality of segments. Message identifiers associated with the plurality of segments are then plaeed into the message queue. The transport system then selects one of the plurality of segments from the message queue for transport and determines if the selected segment has previously been transported. the selected segment is reduced to a differenced communication format based upon whether it has previously been transported to provide a reduced segment and emitted. After verifying that the differenced communication format of the reduced segment was known to the destination computer, the selected segment is dequeued and placed in a differencing cache if it has not previously been transported. The message identifiers associated with the plurality of segments may be the associated segments themselves or they may be pointers to a location in memory where associated segments can be found.
In one embodiment of the present invention, the system determines if the segment has previously been transported and plaecd in the differencing cache by first calculating an identifier for the selected segment such as a data signature based on the content of the segment (e.g. a CRC). Based on the calculated identifier, the system determines if the selected segment corresponds to a segment saved in the differencing cache. Dequeuing a message operations may include placing the identifier of the selected segment in a differencing index and the system determines if the segment has previously been transported by determining if the identifier of the selected segment is in the differencing index. A pointer may also be placed in the differencing index enabling the selected segment to be located in the differencing cache. The differencing index is preferably associated with a least recently used list and a least recently used segment is removed from the differencing cache and the differencing index if the differencing cache exceeds a size criteria. In one embodiment of the present invention, reducing a segment includes replacing the selected segment with the identifier associated with the selected segment to provide the reduced segment if the selected segment has previously been emitted and providing the selected segment as the reduced segment if the selected segment has not previously been emitted.
In a further embodiment of the present invention, the message queue receives messages from a plurality of data streams and asynchronously emits (transmits) the messages on the external communication link. The messages may be emitted in a first in first out sequence from the message queue and the message queue may have an associated maximum message size.
In another embodiment of the present invention, a message is received from the destination computer indicating whether the destination computer had a segment in memory corresponding to the emitted identifier. The selected segment rather than the identifier associated with the selected segment is emitted if the received message indicates that the destination computer did not have a segment in memory corresponding to the emitted identifier.
In a further embodiment of the segmenting operations of the present invention, an associated type of the data stream is determined. A rule set is selected for segmenting the data stream based on the determined type and the selected rule set is applied to segment the data stream. The associated type of the data stream may be determined based on the data stream. In one embodiment, the associated type is a MIME type.
In another aspect of the present invention, the destination computer receives the emitted reduced segment and reconstructs the selected segment from the received reduced segment. The reconstructed selected segment is placed in a received message queue. The data stream is then reconstructed responsive to the received message queue. The destination computer determines if the reduced segment contains the selected segment and then determines if a segment associated with the reduced segment is available to the destination computer if the reduced segment does not contain the selected segment. The reduced segment is replaced with the segment associated with the reduced segment if available. Otherwise, a message is emitted to the device from which the reduced segment is received requesting transmission of the selected segment if the segment associated with the reduced segment is not available. Data stream reconstruction may include integrating segments from the received message queue into objects. The reconstructed selected segment may be dequeued from the received message queue after providing the reconstructed selected segment to the reconstructing step. The destination computer may provide a differencing index and differencing cache in a manner analogous to that described for the source computer.
In a further aspect of the present invention, a method is provided for transferring a file from a source computer to a destination computer using a message queue. A current version of the file is copied to a file cache responsive to a file transfer request and file segments are defined associated with the current version. The source computer determines if previous versions of the file having associated file segments are available to the source computer and if the destination computer has one of the previous versions of the file available. A first file segment of the current version is compared with the associated file segments of the previous version and a message identifier for the first file segment is placed in the message queue to request transfer of the first file segment if no matching associated file segments of the previous version are located. Otherwise, a match indication is placed in the message queue for transfer if a matching associated file segment is located.
In one embodiment of the file transfer aspects of the present invention, an identifier is calculated for each of the file segments associated with the current version and the calculated identifiers are placed in a segmenting index. The calculated identifiers are compared to a reference identifier based on one of the associated file segments of the previous version and comparison operations continue until either a match is located or all the associated file segments have been tested. Comparing operations are preferably repeated for each of the file segments of the current version to be transferred. In one embodiment, an identifier is calculated for each of the file segments associated with the current version. A segment length for each of the file segments is also calculated and placed in the segmenting index associated with the calculated identifiers.
In a further embodiment of the file transfer aspects of the present invention, an identification of the previous version of the file is transmitted to the destination computer. A message is then received from the destination computer containing an indication of whether the previous version of the file is available to the destination computer. Furthermore, when a plurality of previous versions are available to the source computer, the identifications of the plurality of previous versions are transmitted to the destination computer and the message received from the destination computer contains an indication of one of the plurality of previous versions of the file which is available to the destination computer. This one of the previous version is then used for comparing operations to provide a differenced communication file transfer to the destination computer.
As will be appreciated by those of skill in this art, while the the above described aspects of the present invention have primarily been discussed as methods, they may also be provided as systems or as computer program products.